A Real-Time 3D Interface Using Uncalibrated Cameras

This paper proposes a real-time 3D user interface using multiple possibly uncalibrated cameras. It tracks the user’s pointer in real-time and solves point correspondences across all the cameras. These correspondences form spatiotemporal “traces” that serve as a medium for sketching in a true 3-D space. Alternatively, they may be interpreted as gestures or control information to elicit some particular action(s). Through view synthesis techniques, the system enables the user to change and seemingly manipulate the viewpoint of the virtual scene even in the absence of camera calibration. It also serves as a flexible, intuitive, and portable mixed-reality display system. The proposed system has numerous implications in interaction and design, especially as a general interface for creating and manipulating various forms of 3-D media.